Skip to content

[WIP] - #153

Draft
hunhoffe wants to merge 10 commits into
develfrom
dedupe-sequence-subview
Draft

[WIP]#153
hunhoffe wants to merge 10 commits into
develfrom
dedupe-sequence-subview

Conversation

@hunhoffe

Copy link
Copy Markdown
Collaborator

Not ready for review yet.

PR Merge Checklist

  1. The PR is rebased on the latest devel commit and pointing to devel.
  2. Your PR has been reviewed and approved.
  3. All checks are passing.

atassis and others added 10 commits August 3, 2026 11:30
v1.4.0 carries two breaking changes that the pinned v1.3.5.dev20 predates, so
the version bump alone does not build.

mlir-aie #3387 reworked the IRON Runtime into a callback body. Runtime() plus
'with rt.sequence(...)' is gone; the constructor now takes (seq_fn, fn_args),
fill/drain moved onto the ObjectFifo handle, workers moved to Program(workers=),
task_group()/finish_task_group() became TaskGroup()/tg.finish(), set_barrier()
became barrier.set(), inline_ops() became a direct call, enable_trace() moved to
Program and sync_parameters() to module scope. The per-transfer tile= argument is
now bound on .prod()/.cons(), since placement is a property of the handle.

mlir-aie #3364 unified aiecc's output selection under --get-<name>, removing
--aie-generate-xclbin, --aie-generate-npu-insts and --no-compile-host. Asking
only for the insts is what --no-compile used to mean, so that flag goes away
rather than being renamed.

Two spots needed more than a rename. gemm returns tensor access patterns that are
recorded while the sequence body runs, and the body now runs at resolve_program()
rather than at construction, so the program is resolved before the taps are read.
mem_copy guarded rt.start on 'not bypass', which becomes a conditional workers=
argument.

Verified on Strix (npu2): the axpy suite passes 20/20 on device.
…0.2026081401

Builds on #145's v1.4.0 migration. Fixes further upstream breaking changes:
- aiecc flipped its Peano/Chess default; --no-xchesscc/--no-xbridge no longer
  exist, so drop them (Peano is now the implicit default)
- XRTTensor's residency tracking was reworked into a Storage/coherence-map
  model; XRTSubBuffer now shares its parent's _storage instead of
  bootstrapping incompatible state
- drop --dynamic-objFifos everywhere it was passed (default upstream now)

Also fixes iron/operators/__init__.py unconditionally importing the
stream-dse-backed operator, which broke its documented self-skip behavior
when onnx/stream-dse aren't installed.

Verified: 146/146 fast + 2830/2830 extensive operator tests pass on NPU2
hardware (some extensive failures under -n auto were pre-existing xdist
build-directory races, confirmed passing when rerun serially).

Co-Authored-By: Claude <noreply@anthropic.com>
mlir-aie's nightly (1.4.2.dev12) added a first-class subview() API backed by
a shared storage/coherence map between a view and its parent. This directly
supersedes XRTSubBuffer, which existed only because that capability didn't
exist upstream (per its own TODO). Deleting it also fixes a real bug it had:
XRTSubBuffer.to() always resynced the *entire* parent buffer, which could
clobber a sibling sub-view whose fresh device data hadn't been read back yet.
subview()'s shared coherence map tracks dirtiness per byte range instead, so
each view's .to() only moves what it needs to.

Converts all six call sites (iron/common/sequence.py, and four in
llama_npu.py) from XRTSubBuffer(...)/XRTSubBuffer.from_parent(...) to
parent.subview(offset_bytes, shape, dtype), then deletes the class.

Verified: 146/146 fast + 2830/2830 extensive operator tests pass on NPU2
hardware. llama_npu.py's call sites are syntax/import-checked only; no model
weights available locally to run it end-to-end.

Co-Authored-By: Claude <noreply@anthropic.com>
…_arch()

get_kernel_dir() reimplemented the device-arch-to-kernel-dir-string mapping
that mlir-aie's aie.utils.compile.utils.resolve_target_arch() now provides.
Keep our own no-arg auto-detect default (resolve_target_arch(None) returns
"aie2" unconditionally rather than checking the current device, which would
silently regress every no-arg call site on NPU2 hardware), but delegate the
actual device->arch-string logic once a concrete device is known.

Verified: 2830/2830 operator tests pass on NPU2 hardware.

Co-Authored-By: Claude <noreply@anthropic.com>
Replaces the hand-built clang++/xchesscc invocation in
KernelCompilationRule.compile() with aie.utils.compile.utils's
compile_cxx_core_function(), wrapped in PythonCallbackCompilationCommand
(which exists precisely for this: an arbitrary Python callable deferred
into the same plan/execute contract as ShellCompilationCommand).

This looked architecturally blocked at first (IRON's compile() methods
return commands for a separate execute() step, while upstream's function
runs synchronously), but IRON's execute() is already fully sequential in
Python -- the only real parallelism is aiecc's own internal -j flag inside
a single invocation, which is unaffected here since this rule only compiles
individual kernel objects, not the aiecc/xclbin step. Nothing is lost by
switching to a blocking call under a deferred wrapper.

peano_dir/mlir_aie_dir were already always sourced from
aie.utils.config.peano_install_dir()/root_path() (iron/common/context.py),
the same config module the upstream function reads internally, so there's
no override capability to lose. xchesscc_wrapper resolves via `shutil.which`
upstream instead of an absolute path IRON constructed itself, but the wheel
install puts it on PATH (ironenv/bin/xchesscc_wrapper), so this is a
non-issue in the standard install.

-Wno-missing-template-arg-list-after-template-kw is kept as an IRON-side
extra flag (Peano-only, matching prior behavior) since it's specific to our
kernel sources and upstream's default flag list doesn't include it.

Symbol renaming/prefixing for operator fusion (_rename_symbols/
_prefix_symbols) stays IRON-side; upstream has no equivalent multi-symbol
bulk-prefix operation.

Verified: 146/146 fast + 2830/2830 extensive operator tests pass on NPU2
hardware, including a from-scratch rebuild (rm -rf build/) to exercise every
kernel compile through the new path. The use_chess=True path is unexercised
by any test (also true before this change).

Co-Authored-By: Claude <noreply@anthropic.com>
Replaces the hand-built aiecc invocation in AieccXclbinInstsCompilationRule
and AieccFullElfCompilationRule with aie.utils.compile.utils's
compile_mlir_module(), wrapped in PythonCallbackCompilationCommand (same
pattern as the prior KernelCompilationRule change).

Two real gaps had to be worked around, not just papered over:

- compile_mlir_module() always names its own copy of the MLIR source
  "aie.mlir" inside a work_dir it controls, rather than accepting an existing
  file path. IRON's build previously kept every artifact's .mlir file (and
  aiecc's ".prj" companion) at a distinctive flat filename in one shared
  build/ directory. Introduced _aiecc_work_dir(), giving each MLIR source its
  own "<name>.mlir.d/" subdirectory, and updated the one place that depended
  on the old "<name>.mlir.prj" convention (OperatorSequence's params.txt
  lookup in sequence.py) to derive the new path the same way.

- aiecc resolves an MLIR module's relative kernel-object references (e.g.
  link_with = "axpy.o") against that work_dir, not against the flat build_dir
  where KernelCompilationRule/ArchiveCompilationRule actually produced them.
  Added _link_build_outputs_into() to symlink the build directory's existing
  outputs into each artifact's work_dir before compiling, rather than
  reworking where kernel objects get built.

Known, accepted regression: compile_mlir_module() resolves aiecc via
aie.utils.compile.utils's own config.aiecc_path(), which does not honor the
AIECC_PATH env var IRON previously supported for pointing a build at a
locally-built aiecc without reinstalling the wheel. No test exercises this,
but it's a real dev-workflow feature being dropped, not just an implementation
detail.

peano_dir/mlir_aie_dir/build_dir are no longer threaded into
AieccCompilationRule's constructor (dead after this change), simplifying the
context.py call sites accordingly.

Verified: 146/146 fast + 2830/2830 extensive operator tests pass on NPU2
hardware, from a clean build/ (rm -rf) to exercise every aiecc invocation
through the new path, and again from a warm cache.

Co-Authored-By: Claude <noreply@anthropic.com>
Bumps to the latest nightly wheels as of 2026-08-20 (mlir-aie
1.4.2.dev16+g7e00b57, llvm-aie 22.0.0.2026082001+84660bc3), up from
1.4.2.dev12 / 2026081401.

Verified: 725/750 fast (non-extensive) operator/infra tests pass on
NPU2 hardware. The 25 failures are the pre-existing fused-dispatch
aiebu-asm gap (XRT 2.20.0 lacks aiebu-asm on PATH; needs >=2.21),
unrelated to this bump.

Co-Authored-By: Claude <noreply@anthropic.com>
SequenceFullELFCallable.params looked for params.txt under a nested
"aie.mlir.prj" subdirectory of the aiecc work dir. That convention matched
aiecc's *default* project-dir naming (used when --tmpdir is omitted), but
compile_mlir_module() (introduced in 4f50d26) always passes --tmpdir
explicitly, in which case aiecc treats that directory as the project dir
itself rather than nesting another ".prj" folder under it. Confirmed against
a fresh build/<name>.mlir.d/ directory: aie.mlir, bif_main.bif, cdo_main/ all
land directly in it, not in a nested aie.mlir.prj/.

This is the only caller of ParameterScratchpad in the tree, and only
exercised by the llama_3.2_1b decode path (cache_offset/softmax_vector_size),
so it wasn't caught by the 146/146 fast + 2830/2830 extensive operator runs.

Co-Authored-By: Claude <noreply@anthropic.com>
3cded00 replaced IRON's hand-rolled XRTSubBuffer with upstream
NpuTensor.subview() for the XRT dispatch path (SequenceXclbinCallable), but
left one spot behind: SequenceReferenceCallable's pure-CPU path still built
its sliced sub-buffers by hand, constructing a bare CPUOnlyTensor and
overwriting its _data/_shape fields directly with a numpy slice.

CPUOnlyTensor is itself an NpuTensor subclass and already implements
_subview(), so this was reimplementing, by hand, what parent.subview(...)
already does for this backend -- minus the 64-byte coherence-granule
alignment check subview() enforces "for every backend, including the
CPU-only one that has no coherence concern of its own... so a layout
validated against the test backend stays valid on a device" (per its
docstring). That check previously only fired on the XRT path; a layout
that would fail on hardware could silently pass under dispatch="reference".

Once both _make_subbuffer overrides just called .subview() with the same
arguments, the hook itself stopped pulling weight: inlined the .subview()
call directly into the shared _PerBufferCallable._resolve_buffer and
deleted _make_subbuffer from both subclasses and the base class. Only
_make_buffer (XRTTensor vs. CPUOnlyTensor construction) remains as the
per-backend hook, which is genuine backend variation.

fuse_mlir() (iron/common/compilation/sequence.py) is out of scope here: it
generates memref.subview/reinterpret_cast MLIR directly to implement IRON's
own operator-fusion runlist calling convention, which has no upstream
equivalent to delegate to.

dispatch="reference" had no test coverage at all in
iron/tests/infrastructure/sequence.py before this change, and specifically
none exercising slice_info/explicit_buffer_sizes resolution on the CPU
backend -- the only real caller of sliced buffers (llama's per-head
attention slicing) only runs the XRT path. Added
test_reference_dispatch_resolves_sliced_buffer to close that gap.

Verified: 755/755 fast (non-extensive) operator/infra/application tests
pass on NPU2 hardware (24 llama tests skip, pre-existing XRT 2.20.0 gap),
including the new test and the full iron/tests/infrastructure/sequence.py
suite (80/80). Extensive suite not re-run: its markers cover per-operator
kernel parameter sweeps, orthogonal to this sequence-dispatch change.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

ac74085 (2026_08_24_18_10_18)

IRON - CI Summary

Small

iron/operators/axpy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] - - 323.58
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] - - 278.64
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] - - 352.64
iron/operators/dequant
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] - - 304.96
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] - - 460.80
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] - - 440.10
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] - - 436.90
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] - - 351.24
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] - - 801.12
iron/operators/elementwise_add
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] - - 361.30
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] - - 271.00
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] - - 534.74
iron/operators/elementwise_mul
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] - - 309.02
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] - - 319.54
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] - - 384.22
iron/operators/gelu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 330.80
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - 674.66
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 399.60
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - 696.24
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 411.92
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - 558.32
iron/operators/gemm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] - - 678.98
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] - - 687.56
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - 81985.78
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - 24833.90
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] - - 4071.28
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] - - 6470.62
iron/operators/gemv
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - 0.08
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - 3.62
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] - - 6.72
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] - - 9.21
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - 3.75
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] - - 7.10
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] - - 10.04
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2] - - 2.79
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2] - - 0.27
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4] - - 0.50
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8] - - 1.76
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - -
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - -
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - -
iron/operators/layer_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 468.02
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - 398.96
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 355.06
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - 690.78
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 367.94
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - 433.00
iron/operators/leaky_relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01] - - 398.54
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1] - - 448.58
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25] - - 453.18
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01] - - 433.66
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01] - - 430.98
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01] - - 364.54
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01] - - 383.74
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01] - - 448.90
iron/operators/mem_copy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] - - 282.66
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] - - 374.70
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] - - 373.98
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] - - 431.86
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] - - 399.66
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] - - 543.82
iron/operators/relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 672.46
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - 417.10
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 632.68
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - 398.08
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 541.96
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - 399.92
iron/operators/rms_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False] - - 413.70
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True] - - 820.66
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False] - - 424.38
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True] - - 314.74
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False] - - 408.42
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True] - - 467.32
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False] - - 431.08
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True] - - 600.70
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] - - 343.12
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True] - - 598.12
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False] - - 979.60
iron/operators/rope
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0] - - 368.40
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0] - - 368.96
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0] - - 713.82
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0] - - 326.70
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0] - - 355.26
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0] - - 386.50
iron/operators/sigmoid
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 275.32
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - 276.08
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 433.16
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - 395.00
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 493.66
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - 441.70
iron/operators/silu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 298.70
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 353.96
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 419.98
iron/operators/softmax
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024] - - 549.74
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048] - - 488.02
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512] - - 405.74
iron/operators/swiglu_decode
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] - - 11700.74
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] - - 11559.62
iron/operators/swiglu_prefill
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] - - 18412.11
iron/operators/tanh
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - 304.36
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - 347.50
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - 356.00
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - 480.74
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - 455.86
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - 481.78
iron/operators/transpose
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1] - - 512.78
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2] - - 1146.82
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1] - - 686.02
Phoenix - Small

IRON

Tested on 2026_08_24_18_10_18 at commit ac74085.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5323.580.04n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5278.640.05n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5352.640.04n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5304.960.02n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5460.800.01n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5440.100.01n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5436.900.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5351.240.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5801.120.01n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5361.300.04n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5271.000.05n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5534.740.03n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5309.020.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5319.540.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5384.220.04n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5330.800.03n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5674.660.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5399.600.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5696.240.01n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5411.920.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5558.320.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5678.980.3715.85
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5687.560.4619.82
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/581985.780.31209.58
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/524833.901.01691.89
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/54071.282.39627.75
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/56470.620.2010.66
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.080.08
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.623.62
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a6.726.71
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a9.219.20
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.753.75
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a7.107.09
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a10.0410.03
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]✅ 5/5n/a2.792.79
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]✅ 5/5n/a0.270.27
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]✅ 5/5n/a0.500.50
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]✅ 5/5n/a1.761.74
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]❌ 0/5n/an/an/a
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]❌ 0/5n/an/an/a
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]❌ 0/5n/an/an/a
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5468.020.02n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5398.960.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5355.060.03n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5690.780.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5367.940.03n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5433.000.02n/a
iron/operators/leaky_relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]✅ 5/5398.540.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]✅ 5/5448.580.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]✅ 5/5453.180.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]✅ 5/5433.660.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]✅ 5/5430.980.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]✅ 5/5364.540.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]✅ 5/5383.740.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]✅ 5/5448.900.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5282.660.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5374.700.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5373.980.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5431.860.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5399.660.02n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5543.820.02n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5672.460.02n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5417.100.02n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5632.680.02n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5398.080.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5541.960.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5399.920.02n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]✅ 5/5413.700.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]✅ 5/5820.660.03n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]✅ 5/5424.380.02n/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]✅ 5/5314.740.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]✅ 5/5408.420.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]✅ 5/5467.320.03n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]✅ 5/5431.080.02n/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]✅ 5/5600.700.03n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]✅ 5/5343.120.03n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]✅ 5/5598.120.02n/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]✅ 5/5979.600.01n/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5368.400.29n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5368.960.29n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5713.820.25n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5326.700.26n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5355.260.23n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5386.500.25n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5275.320.03n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5276.080.03n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5433.160.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5395.000.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5493.660.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5441.700.02n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5298.700.03n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5353.960.03n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5419.980.02n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5549.740.24n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5488.020.30n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5405.740.34n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/511700.740.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/511559.620.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/518412.110.12n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5304.360.03n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5347.500.03n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5356.000.03n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5480.740.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5455.860.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5481.780.02n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5512.781.18n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/51146.821.66n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5686.020.95n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (+12.79%)0.04 (+19.82%)0.04 (+6.44%)0.02 (+21.41%)0.01 (-8.88%)500.20 (-17.64%)323.58 (-20.55%)277.70 (-6.06%)240.90 (-11.34%)107.89 (-34.91%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)607.30 (n/a)407.28 (n/a)295.60 (n/a)271.70 (n/a)165.76 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.06 (+6.80%)0.05 (+11.98%)0.05 (+7.30%)0.03 (+35.44%)0.01 (-16.00%)429.10 (-26.16%)278.64 (-15.28%)245.70 (-6.79%)220.60 (-6.41%)85.20 (-40.93%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)581.10 (n/a)328.90 (n/a)263.60 (n/a)235.70 (n/a)144.22 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (-3.78%)0.04 (+24.52%)0.04 (+58.37%)0.02 (+1.67%)0.01 (-10.73%)636.00 (-1.64%)352.64 (-20.34%)294.80 (-36.85%)249.20 (+3.92%)159.57 (+1.13%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)646.60 (n/a)442.68 (n/a)466.80 (n/a)239.80 (n/a)157.79 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-18.94%)0.02 (+19.03%)0.02 (+13.25%)0.01 (+64.83%)0.00 (-43.47%)491.50 (-39.33%)304.96 (-29.65%)261.00 (-11.71%)250.70 (+23.38%)104.58 (-58.31%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)810.10 (n/a)433.48 (n/a)295.60 (n/a)203.20 (n/a)250.85 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-9.01%)0.01 (-25.51%)0.01 (-37.23%)0.01 (-25.35%)0.00 (+1.28%)598.80 (+33.96%)460.80 (+37.71%)480.50 (+59.32%)248.70 (+9.90%)137.62 (+37.82%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)447.00 (n/a)334.62 (n/a)301.60 (n/a)226.30 (n/a)99.85 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-4.85%)0.01 (-14.74%)0.01 (-17.67%)0.01 (-38.90%)0.01 (+24.53%)811.80 (+63.67%)440.10 (+31.49%)353.10 (+21.47%)235.50 (+5.09%)234.58 (+112.31%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)496.00 (n/a)334.70 (n/a)290.70 (n/a)224.10 (n/a)110.49 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (+6.36%)0.01 (-14.61%)0.01 (-27.97%)0.01 (+7.79%)0.00 (+0.98%)545.50 (-7.23%)436.90 (+16.16%)420.70 (+38.84%)269.00 (-5.98%)112.23 (-11.65%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)588.00 (n/a)376.12 (n/a)303.00 (n/a)286.10 (n/a)127.02 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-10.66%)0.02 (-10.08%)0.02 (+6.28%)0.01 (-5.55%)0.00 (-2.51%)494.40 (+5.89%)351.24 (+12.27%)289.40 (-5.92%)248.20 (+11.95%)109.73 (+16.04%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)466.90 (n/a)312.86 (n/a)307.60 (n/a)221.70 (n/a)94.56 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-15.53%)0.01 (-16.15%)0.01 (-0.18%)0.00 (-75.76%)0.01 (+17.75%)2433.10 (+312.60%)801.12 (+91.09%)453.60 (+0.18%)289.50 (+18.40%)916.25 (+550.70%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)589.70 (n/a)419.24 (n/a)452.80 (n/a)244.50 (n/a)140.81 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)603.10 (n/a)361.30 (n/a)267.50 (n/a)240.60 (n/a)160.69 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)449.10 (n/a)271.00 (n/a)229.60 (n/a)179.60 (n/a)104.49 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)1052.10 (n/a)534.74 (n/a)486.10 (n/a)236.70 (n/a)308.39 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)469.40 (n/a)309.02 (n/a)283.50 (n/a)244.90 (n/a)92.89 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)544.20 (n/a)319.54 (n/a)289.00 (n/a)204.10 (n/a)133.16 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.06 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)624.70 (n/a)384.22 (n/a)295.30 (n/a)223.20 (n/a)170.28 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)559.90 (n/a)330.80 (n/a)261.20 (n/a)256.30 (n/a)130.43 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.01 (n/a)1935.70 (n/a)674.66 (n/a)428.90 (n/a)241.30 (n/a)714.88 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)552.70 (n/a)399.60 (n/a)407.60 (n/a)230.10 (n/a)143.51 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)1057.80 (n/a)696.24 (n/a)570.60 (n/a)276.60 (n/a)333.10 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)574.90 (n/a)411.92 (n/a)384.70 (n/a)251.00 (n/a)143.45 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)724.90 (n/a)558.32 (n/a)559.20 (n/a)431.10 (n/a)109.04 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.62 (+17.23%)0.37 (-9.80%)0.32 (-22.31%)0.20 (-27.45%)0.16 (+63.43%)1086.30 (+37.84%)678.98 (+20.54%)697.60 (+28.71%)356.30 (-14.70%)267.91 (+86.73%)26.49 (+17.23%)15.85 (-9.80%)13.53 (-22.31%)8.69 (-27.45%)6.66 (+63.43%)
cdc48e9 — 2026-08-03 16:32:080.53 (n/a)0.41 (n/a)0.41 (n/a)0.28 (n/a)0.10 (n/a)788.10 (n/a)563.26 (n/a)542.00 (n/a)417.70 (n/a)143.47 (n/a)22.60 (n/a)17.57 (n/a)17.41 (n/a)11.97 (n/a)4.07 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.79 (+64.86%)0.46 (+20.62%)0.47 (+33.31%)0.13 (-54.93%)0.23 (+175.35%)1739.90 (+121.90%)687.56 (+15.03%)469.70 (-24.98%)280.30 (-39.34%)594.37 (+343.50%)33.67 (+64.86%)19.82 (+20.62%)20.09 (+33.31%)5.42 (-54.93%)10.01 (+175.35%)
cdc48e9 — 2026-08-03 16:32:080.48 (n/a)0.39 (n/a)0.35 (n/a)0.28 (n/a)0.09 (n/a)784.10 (n/a)597.72 (n/a)626.10 (n/a)462.10 (n/a)134.02 (n/a)20.42 (n/a)16.43 (n/a)15.07 (n/a)12.04 (n/a)3.64 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.31 (+2.04%)0.31 (+0.60%)0.31 (+0.32%)0.30 (-0.09%)0.00 (+123.32%)83428.70 (+0.09%)81985.78 (-0.58%)82022.00 (-0.32%)80475.80 (-2.00%)1096.95 (+118.75%)213.48 (+2.04%)209.58 (+0.60%)209.45 (+0.32%)205.92 (-0.09%)2.81 (+123.32%)
cdc48e9 — 2026-08-03 16:32:080.31 (n/a)0.31 (n/a)0.31 (n/a)0.30 (n/a)0.00 (n/a)83350.90 (n/a)82467.46 (n/a)82282.50 (n/a)82115.60 (n/a)501.45 (n/a)209.22 (n/a)208.33 (n/a)208.79 (n/a)206.12 (n/a)1.26 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:581.03 (+0.72%)1.01 (+1.31%)1.02 (+0.77%)0.99 (+3.64%)0.01 (-46.04%)25395.60 (-3.51%)24833.90 (-1.33%)24773.20 (-0.77%)24520.60 (-0.72%)340.56 (-48.42%)700.63 (+0.72%)691.89 (+1.31%)693.48 (+0.77%)676.49 (+3.64%)9.38 (-46.04%)
cdc48e9 — 2026-08-03 16:32:081.02 (n/a)1.00 (n/a)1.01 (n/a)0.96 (n/a)0.03 (n/a)26319.80 (n/a)25168.86 (n/a)24964.50 (n/a)24697.80 (n/a)660.22 (n/a)695.60 (n/a)682.95 (n/a)688.17 (n/a)652.74 (n/a)17.38 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:583.83 (+0.02%)2.39 (+0.83%)2.00 (-1.63%)1.34 (-11.85%)1.16 (+27.99%)6012.90 (+13.45%)4071.28 (+8.37%)4022.50 (+1.66%)2105.70 (-0.02%)1855.36 (+51.37%)1003.91 (+0.02%)627.75 (+0.83%)525.52 (-1.63%)351.57 (-11.85%)304.93 (+27.99%)
cdc48e9 — 2026-08-03 16:32:083.83 (n/a)2.37 (n/a)2.04 (n/a)1.52 (n/a)0.91 (n/a)5300.10 (n/a)3756.70 (n/a)3956.90 (n/a)2106.20 (n/a)1225.67 (n/a)1003.68 (n/a)622.60 (n/a)534.24 (n/a)398.85 (n/a)238.25 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.24 (-21.07%)0.20 (-6.92%)0.21 (+7.61%)0.15 (-18.44%)0.03 (-29.54%)8539.00 (+22.61%)6470.62 (+6.78%)5894.50 (-7.07%)5296.70 (+26.69%)1270.43 (+15.95%)12.67 (-21.07%)10.66 (-6.92%)11.39 (+7.61%)7.86 (-18.44%)1.84 (-29.54%)
cdc48e9 — 2026-08-03 16:32:080.30 (n/a)0.21 (n/a)0.20 (n/a)0.18 (n/a)0.05 (n/a)6964.20 (n/a)6059.96 (n/a)6342.80 (n/a)4180.90 (n/a)1095.64 (n/a)16.05 (n/a)11.45 (n/a)10.58 (n/a)9.64 (n/a)2.62 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.12 (n/a)0.08 (n/a)0.07 (n/a)0.06 (n/a)0.03 (n/a)0.12 (n/a)0.08 (n/a)0.07 (n/a)0.06 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:583.80 (n/a)3.62 (n/a)3.66 (n/a)3.34 (n/a)0.18 (n/a)3.80 (n/a)3.62 (n/a)3.65 (n/a)3.34 (n/a)0.18 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:587.55 (+33.30%)6.72 (+28.74%)7.05 (+33.57%)5.73 (+20.61%)0.84 (+89.95%)7.55 (+33.30%)6.71 (+28.74%)7.05 (+33.57%)5.73 (+20.61%)0.84 (+89.95%)
cdc48e9 — 2026-08-03 16:32:085.67 (n/a)5.22 (n/a)5.28 (n/a)4.75 (n/a)0.44 (n/a)5.66 (n/a)5.21 (n/a)5.28 (n/a)4.75 (n/a)0.44 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:5813.85 (+7.19%)9.21 (-7.01%)8.44 (-15.19%)7.55 (+17.71%)2.63 (+5.37%)13.84 (+7.19%)9.20 (-7.01%)8.43 (-15.19%)7.54 (+17.71%)2.63 (+5.38%)
cdc48e9 — 2026-08-03 16:32:0812.92 (n/a)9.90 (n/a)9.95 (n/a)6.41 (n/a)2.49 (n/a)12.91 (n/a)9.90 (n/a)9.95 (n/a)6.41 (n/a)2.49 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:583.82 (n/a)3.75 (n/a)3.78 (n/a)3.65 (n/a)0.08 (n/a)3.82 (n/a)3.75 (n/a)3.78 (n/a)3.65 (n/a)0.08 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:587.74 (+2.47%)7.10 (+2.76%)7.12 (+3.67%)6.58 (+9.04%)0.51 (-21.10%)7.74 (+2.47%)7.09 (+2.76%)7.12 (+3.67%)6.58 (+9.04%)0.51 (-21.10%)
cdc48e9 — 2026-08-03 16:32:087.56 (n/a)6.91 (n/a)6.87 (n/a)6.04 (n/a)0.65 (n/a)7.55 (n/a)6.90 (n/a)6.87 (n/a)6.03 (n/a)0.65 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:5814.15 (+1.85%)10.04 (-18.19%)9.38 (-25.08%)8.02 (-15.69%)2.37 (+30.36%)14.14 (+1.85%)10.03 (-18.19%)9.38 (-25.08%)8.01 (-15.69%)2.37 (+30.36%)
cdc48e9 — 2026-08-03 16:32:0813.89 (n/a)12.27 (n/a)12.52 (n/a)9.51 (n/a)1.82 (n/a)13.89 (n/a)12.27 (n/a)12.52 (n/a)9.50 (n/a)1.82 (n/a)

test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:583.17 (+14.83%)2.79 (+89.48%)2.80 (+123.70%)2.33 (+139.60%)0.31 (-58.32%)3.17 (+14.83%)2.79 (+89.48%)2.80 (+123.70%)2.32 (+139.60%)0.30 (-58.32%)
cdc48e9 — 2026-08-03 16:32:082.76 (n/a)1.47 (n/a)1.25 (n/a)0.97 (n/a)0.73 (n/a)2.76 (n/a)1.47 (n/a)1.25 (n/a)0.97 (n/a)0.73 (n/a)

test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.50 (+10.86%)0.27 (+18.41%)0.26 (+39.85%)0.09 (+19.45%)0.18 (+3.93%)0.50 (+10.86%)0.27 (+18.41%)0.25 (+39.85%)0.09 (+19.45%)0.17 (+3.93%)
cdc48e9 — 2026-08-03 16:32:080.45 (n/a)0.23 (n/a)0.18 (n/a)0.07 (n/a)0.17 (n/a)0.45 (n/a)0.22 (n/a)0.18 (n/a)0.07 (n/a)0.17 (n/a)

test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:580.86 (+96.62%)0.50 (+66.52%)0.41 (+42.47%)0.36 (+354.97%)0.21 (+40.27%)0.85 (+96.62%)0.50 (+66.52%)0.40 (+42.47%)0.36 (+354.97%)0.20 (+40.27%)
cdc48e9 — 2026-08-03 16:32:080.44 (n/a)0.30 (n/a)0.29 (n/a)0.08 (n/a)0.15 (n/a)0.43 (n/a)0.30 (n/a)0.28 (n/a)0.08 (n/a)0.14 (n/a)

test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
ac74085 — 2026-08-24 18:05:582.66 (-12.30%)1.76 (-7.84%)1.79 (-2.73%)0.44 (-57.32%)0.84 (+17.67%)2.62 (-12.30%)1.74 (-7.84%)1.76 (-2.73%)0.43 (-57.32%)0.83 (+17.67%)
cdc48e9 — 2026-08-03 16:32:083.03 (n/a)1.91 (n/a)1.84 (n/a)1.03 (n/a)0.72 (n/a)2.98 (n/a)1.88 (n/a)1.81 (n/a)1.01 (n/a)0.71 (n/a)

test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

No metrics available.

test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

No metrics available.

test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

No metrics available.

iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)635.90 (n/a)468.02 (n/a)489.20 (n/a)216.90 (n/a)153.16 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)549.30 (n/a)398.96 (n/a)352.90 (n/a)289.10 (n/a)121.28 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)498.00 (n/a)355.06 (n/a)297.00 (n/a)231.50 (n/a)126.29 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.01 (n/a)2071.70 (n/a)690.78 (n/a)338.60 (n/a)256.60 (n/a)778.74 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)590.90 (n/a)367.94 (n/a)273.00 (n/a)239.10 (n/a)163.79 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)587.00 (n/a)433.00 (n/a)378.30 (n/a)298.80 (n/a)118.48 (n/a)
iron/operators/leaky_relu

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+22.17%)0.02 (+31.58%)0.03 (+82.69%)0.01 (-10.06%)0.01 (+44.89%)703.90 (+11.20%)398.54 (-18.49%)294.00 (-45.25%)245.20 (-18.16%)198.11 (+25.27%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)633.00 (n/a)488.92 (n/a)537.00 (n/a)299.60 (n/a)158.15 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (-6.78%)0.02 (-31.65%)0.02 (-47.75%)0.02 (+0.25%)0.01 (-3.86%)545.60 (-0.26%)448.58 (+44.97%)488.20 (+91.38%)230.70 (+7.25%)127.72 (-6.15%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)547.00 (n/a)309.42 (n/a)255.10 (n/a)215.10 (n/a)136.08 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-24.14%)0.02 (-31.17%)0.02 (-32.22%)0.01 (-40.26%)0.01 (-21.37%)670.20 (+67.38%)453.18 (+47.83%)386.90 (+47.56%)313.70 (+31.86%)140.84 (+74.80%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)400.40 (n/a)306.56 (n/a)262.20 (n/a)237.90 (n/a)80.57 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (+10.99%)0.02 (-15.09%)0.02 (-42.44%)0.01 (-15.95%)0.01 (+32.05%)570.90 (+18.99%)433.66 (+24.74%)497.10 (+73.75%)203.70 (-9.87%)151.99 (+31.53%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)479.80 (n/a)347.66 (n/a)286.10 (n/a)226.00 (n/a)115.56 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-32.47%)0.02 (-27.42%)0.02 (-25.60%)0.01 (-18.30%)0.00 (-57.82%)562.60 (+22.41%)430.98 (+29.35%)404.10 (+34.39%)330.70 (+48.10%)86.72 (-24.20%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)459.60 (n/a)333.20 (n/a)300.70 (n/a)223.30 (n/a)114.40 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+5.07%)0.02 (+22.80%)0.03 (+85.00%)0.02 (+95.10%)0.01 (-28.17%)513.20 (-48.75%)364.54 (-31.30%)304.40 (-45.94%)244.60 (-4.82%)122.17 (-59.71%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)1001.30 (n/a)530.62 (n/a)563.10 (n/a)257.00 (n/a)303.27 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+16.69%)0.02 (+9.55%)0.02 (+16.62%)0.02 (+13.17%)0.01 (+14.37%)537.10 (-11.63%)383.74 (-8.20%)329.10 (-14.27%)245.50 (-14.28%)130.45 (-6.53%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)607.80 (n/a)418.02 (n/a)383.90 (n/a)286.40 (n/a)139.56 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-9.82%)0.02 (+0.40%)0.02 (+17.93%)0.01 (+6.86%)0.01 (-18.29%)632.00 (-6.43%)448.90 (-2.90%)410.40 (-15.21%)288.60 (+10.91%)136.92 (-11.37%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)675.40 (n/a)462.30 (n/a)484.00 (n/a)260.20 (n/a)154.49 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-21.89%)0.03 (+20.55%)0.03 (+51.76%)0.02 (+459.10%)0.00 (-71.48%)353.60 (-82.11%)282.66 (-56.57%)279.00 (-34.10%)238.20 (+28.00%)46.25 (-93.84%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.02 (n/a)1977.00 (n/a)650.86 (n/a)423.40 (n/a)186.10 (n/a)750.31 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-2.02%)0.02 (+9.65%)0.03 (+51.57%)0.01 (+0.64%)0.01 (-9.72%)598.20 (-0.65%)374.70 (-11.15%)274.40 (-34.02%)246.20 (+2.07%)159.10 (-9.79%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)602.10 (n/a)421.72 (n/a)415.90 (n/a)241.20 (n/a)176.36 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+4.47%)0.02 (+17.25%)0.02 (-0.87%)0.02 (+17.29%)0.01 (+14.27%)468.40 (-14.74%)373.98 (-13.77%)439.10 (+0.90%)241.00 (-4.29%)111.90 (-0.51%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)549.40 (n/a)433.68 (n/a)435.20 (n/a)251.80 (n/a)112.47 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-4.78%)0.02 (-6.70%)0.02 (+17.41%)0.01 (-19.62%)0.01 (-14.25%)689.50 (+24.41%)431.86 (+6.62%)396.90 (-14.83%)248.10 (+5.04%)177.80 (+15.86%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)554.20 (n/a)405.06 (n/a)466.00 (n/a)236.20 (n/a)153.47 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+8.52%)0.02 (-2.65%)0.02 (-28.84%)0.02 (+11.35%)0.01 (+11.15%)533.60 (-10.20%)399.66 (+2.82%)456.20 (+40.54%)265.90 (-7.87%)118.57 (-9.56%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)594.20 (n/a)388.70 (n/a)324.60 (n/a)288.60 (n/a)131.10 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (+6.52%)0.02 (+8.27%)0.02 (+68.08%)0.01 (-43.83%)0.01 (+15.31%)1283.40 (+78.05%)543.82 (+9.44%)345.40 (-40.51%)267.30 (-6.14%)424.32 (+116.74%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)720.80 (n/a)496.92 (n/a)580.60 (n/a)284.80 (n/a)195.77 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-5.29%)0.02 (-5.02%)0.02 (-9.89%)0.01 (+0.74%)0.01 (-17.15%)579.10 (-0.74%)413.70 (+2.33%)436.00 (+10.97%)276.20 (+5.58%)125.76 (-12.92%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)583.40 (n/a)404.30 (n/a)392.90 (n/a)261.60 (n/a)144.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.05 (+7.23%)0.03 (+8.65%)0.02 (-2.16%)0.00 (-72.69%)0.02 (+69.23%)2539.80 (+266.18%)820.66 (+63.69%)510.80 (+2.20%)251.30 (-6.75%)970.06 (+536.38%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)693.60 (n/a)501.36 (n/a)499.80 (n/a)269.50 (n/a)152.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (+24.50%)0.02 (+2.88%)0.02 (+12.60%)0.01 (-7.48%)0.01 (+32.88%)594.80 (+8.09%)424.38 (+0.35%)399.00 (-11.20%)218.50 (-19.70%)141.96 (+9.59%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)550.30 (n/a)422.88 (n/a)449.30 (n/a)272.10 (n/a)129.54 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (+8.84%)0.03 (+28.96%)0.03 (+45.07%)0.02 (+31.99%)0.01 (-12.86%)433.80 (-24.23%)314.74 (-24.26%)299.80 (-31.06%)265.80 (-8.12%)68.29 (-36.92%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)572.50 (n/a)415.56 (n/a)434.90 (n/a)289.30 (n/a)108.25 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (-6.25%)0.02 (-6.58%)0.02 (-43.01%)0.02 (+17.42%)0.01 (+9.78%)535.60 (-14.84%)408.42 (+8.29%)509.90 (+75.46%)233.70 (+6.66%)159.41 (-4.55%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)628.90 (n/a)377.14 (n/a)290.60 (n/a)219.10 (n/a)167.00 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (-3.47%)0.03 (+30.80%)0.02 (+9.38%)0.02 (+217.90%)0.01 (-11.36%)634.80 (-68.54%)467.32 (-42.31%)554.90 (-8.57%)271.30 (+3.59%)177.17 (-74.34%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)2018.00 (n/a)810.12 (n/a)606.90 (n/a)261.90 (n/a)690.59 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (+3.11%)0.02 (-13.84%)0.03 (-15.30%)0.01 (-29.64%)0.01 (+37.25%)762.10 (+42.13%)431.08 (+33.46%)323.70 (+18.05%)208.80 (-3.02%)240.71 (+87.93%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)536.20 (n/a)323.00 (n/a)274.20 (n/a)215.30 (n/a)128.08 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-25.92%)0.03 (-5.08%)0.03 (+6.61%)0.01 (-65.84%)0.01 (-3.54%)1832.80 (+192.73%)600.70 (+51.81%)297.30 (-6.21%)276.90 (+35.01%)688.84 (+279.99%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)626.10 (n/a)395.70 (n/a)317.00 (n/a)205.10 (n/a)181.28 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.04 (+8.61%)0.03 (+31.17%)0.03 (+65.79%)0.02 (+16.86%)0.01 (-2.40%)526.00 (-14.43%)343.12 (-26.04%)316.50 (-39.69%)208.60 (-7.90%)124.55 (-21.49%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)614.70 (n/a)463.90 (n/a)524.80 (n/a)226.50 (n/a)158.65 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.03 (-23.54%)0.02 (-33.91%)0.01 (-35.85%)0.01 (-35.25%)0.01 (-24.20%)869.60 (+54.43%)598.12 (+53.18%)632.50 (+55.86%)321.90 (+30.80%)205.03 (+53.45%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)563.10 (n/a)390.46 (n/a)405.80 (n/a)246.10 (n/a)133.62 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.02 (-47.41%)0.01 (-33.94%)0.01 (-16.26%)0.00 (-64.78%)0.01 (-42.53%)2516.20 (+183.93%)979.60 (+81.23%)618.40 (+19.43%)447.70 (+90.19%)866.37 (+265.65%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)886.20 (n/a)540.54 (n/a)517.80 (n/a)235.40 (n/a)236.94 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.40 (+10.93%)0.29 (+5.96%)0.32 (-6.11%)0.20 (+26.12%)0.09 (-16.98%)499.20 (-20.71%)368.40 (-11.96%)307.70 (+6.51%)247.30 (-9.88%)116.33 (-37.70%)
cdc48e9 — 2026-08-03 16:32:080.36 (n/a)0.27 (n/a)0.34 (n/a)0.16 (n/a)0.10 (n/a)629.60 (n/a)418.44 (n/a)288.90 (n/a)274.40 (n/a)186.73 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.41 (-13.92%)0.29 (-12.05%)0.24 (-33.85%)0.21 (+6.64%)0.09 (-25.35%)473.20 (-6.22%)368.96 (+8.12%)415.70 (+51.16%)240.40 (+16.14%)104.15 (-23.38%)
cdc48e9 — 2026-08-03 16:32:080.47 (n/a)0.33 (n/a)0.36 (n/a)0.19 (n/a)0.12 (n/a)504.60 (n/a)341.26 (n/a)275.00 (n/a)207.00 (n/a)135.94 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.42 (+18.17%)0.25 (+1.14%)0.22 (+16.10%)0.05 (-70.37%)0.15 (+67.23%)2025.90 (+237.54%)713.82 (+58.54%)441.30 (-13.86%)233.60 (-15.36%)748.43 (+395.49%)
cdc48e9 — 2026-08-03 16:32:080.36 (n/a)0.24 (n/a)0.19 (n/a)0.16 (n/a)0.09 (n/a)600.20 (n/a)450.26 (n/a)512.30 (n/a)276.00 (n/a)151.05 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.37 (+27.06%)0.26 (+43.07%)0.26 (+75.17%)0.12 (+16.64%)0.09 (+5.09%)602.20 (-14.27%)326.70 (-32.35%)281.80 (-42.91%)201.50 (-21.29%)157.55 (-22.91%)
cdc48e9 — 2026-08-03 16:32:080.29 (n/a)0.18 (n/a)0.15 (n/a)0.10 (n/a)0.08 (n/a)702.40 (n/a)482.92 (n/a)493.60 (n/a)256.00 (n/a)204.36 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.30 (+3.72%)0.23 (+18.80%)0.26 (+77.68%)0.13 (+1.84%)0.07 (-7.66%)560.40 (-1.80%)355.26 (-17.44%)287.30 (-43.71%)247.60 (-3.58%)128.57 (-10.91%)
cdc48e9 — 2026-08-03 16:32:080.29 (n/a)0.19 (n/a)0.14 (n/a)0.13 (n/a)0.07 (n/a)570.70 (n/a)430.32 (n/a)510.40 (n/a)256.80 (n/a)144.31 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.36 (+20.36%)0.25 (+21.99%)0.27 (+54.10%)0.12 (-24.34%)0.12 (+107.05%)626.10 (+32.17%)386.50 (-0.20%)275.60 (-35.11%)202.30 (-16.92%)219.04 (+147.69%)
cdc48e9 — 2026-08-03 16:32:080.30 (n/a)0.20 (n/a)0.17 (n/a)0.16 (n/a)0.06 (n/a)473.70 (n/a)387.28 (n/a)424.70 (n/a)243.50 (n/a)88.43 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.29 (-44.13%)0.24 (-34.81%)0.26 (-31.37%)0.19 (-20.19%)0.04 (-68.27%)680.60 (+25.29%)549.74 (+42.71%)502.80 (+45.74%)455.20 (+79.00%)93.56 (-29.49%)
cdc48e9 — 2026-08-03 16:32:080.52 (n/a)0.37 (n/a)0.38 (n/a)0.24 (n/a)0.12 (n/a)543.20 (n/a)385.22 (n/a)345.00 (n/a)254.30 (n/a)132.70 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.50 (+98.37%)0.30 (+25.29%)0.26 (+4.79%)0.20 (-3.30%)0.12 (+530.63%)647.60 (+3.42%)488.02 (-12.70%)510.80 (-4.58%)259.80 (-49.58%)140.77 (+201.77%)
cdc48e9 — 2026-08-03 16:32:080.25 (n/a)0.24 (n/a)0.24 (n/a)0.21 (n/a)0.02 (n/a)626.20 (n/a)559.00 (n/a)535.30 (n/a)515.30 (n/a)46.65 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.42 (-18.97%)0.34 (+68.08%)0.39 (+119.13%)0.25 (+280.20%)0.09 (-53.38%)528.60 (-73.70%)405.74 (-63.85%)339.10 (-54.37%)313.00 (+23.42%)111.51 (-85.91%)
cdc48e9 — 2026-08-03 16:32:080.52 (n/a)0.20 (n/a)0.18 (n/a)0.07 (n/a)0.18 (n/a)2009.90 (n/a)1122.34 (n/a)743.10 (n/a)253.60 (n/a)791.49 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.00 (+133.33%)0.00 (+118.18%)0.00 (+200.00%)0.00 (+0.00%)0.00 (+478.79%)22467.98 (+15.24%)11700.74 (-35.36%)7204.63 (-60.46%)5790.77 (-64.65%)7687.92 (+530.03%)
cdc48e9 — 2026-08-03 16:32:080.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)19497.20 (n/a)18101.11 (n/a)18219.31 (n/a)16380.05 (n/a)1220.24 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.00 (+16.67%)0.00 (+48.28%)0.00 (+125.00%)0.00 (+25.00%)0.00 (+8.27%)18085.62 (-21.30%)11559.62 (-32.47%)9249.58 (-51.14%)5729.89 (-15.69%)5293.15 (-16.45%)
cdc48e9 — 2026-08-03 16:32:080.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)22980.49 (n/a)17117.16 (n/a)18929.10 (n/a)6796.32 (n/a)6335.01 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:580.14 (+9.48%)0.12 (+22.95%)0.13 (+44.25%)0.07 (-4.30%)0.03 (+11.83%)29450.73 (+4.48%)18412.11 (-17.61%)16082.92 (-30.71%)14527.02 (-8.64%)6222.92 (+11.84%)
cdc48e9 — 2026-08-03 16:32:080.13 (n/a)0.10 (n/a)0.09 (n/a)0.07 (n/a)0.03 (n/a)28189.10 (n/a)22347.59 (n/a)23211.24 (n/a)15901.35 (n/a)5564.14 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:581.70 (-2.94%)1.18 (+5.82%)1.20 (+27.54%)0.57 (-20.16%)0.45 (-0.82%)912.00 (+25.26%)512.78 (-3.03%)437.40 (-21.58%)308.00 (+3.01%)242.61 (+27.81%)
cdc48e9 — 2026-08-03 16:32:081.75 (n/a)1.12 (n/a)0.94 (n/a)0.72 (n/a)0.45 (n/a)728.10 (n/a)528.78 (n/a)557.80 (n/a)299.00 (n/a)189.82 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:582.62 (-4.78%)1.66 (-13.17%)1.68 (+8.66%)0.30 (-76.35%)0.90 (+34.98%)3506.00 (+322.82%)1146.82 (+90.31%)625.50 (-7.97%)400.50 (+5.01%)1325.66 (+587.59%)
cdc48e9 — 2026-08-03 16:32:082.75 (n/a)1.91 (n/a)1.54 (n/a)1.26 (n/a)0.66 (n/a)829.20 (n/a)602.60 (n/a)679.70 (n/a)381.40 (n/a)192.80 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
ac74085 — 2026-08-24 18:05:581.95 (+10.20%)0.95 (-11.46%)0.69 (-25.76%)0.51 (-35.56%)0.58 (+46.85%)1026.60 (+55.19%)686.02 (+29.40%)756.60 (+34.70%)269.40 (-9.26%)290.56 (+106.43%)
cdc48e9 — 2026-08-03 16:32:081.77 (n/a)1.07 (n/a)0.93 (n/a)0.79 (n/a)0.40 (n/a)661.50 (n/a)530.14 (n/a)561.70 (n/a)296.90 (n/a)140.75 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)
Phoenix - Examples

IRON

Tested on 2026_08_24_18_12_20 at commit ac74085.

Trends:

IRON Trends

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants